WANDERING BUMPS IN STOCHASTIC NEURAL FIELDS 
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Abstract. Wc study the effects of noise on stationary pulse solutions (bumps) in spatially 
extended neural fields. The dynamics of a neural field is described by an intcgrodiffcrential equation 
whose integral term characterizes synaptic interactions between neurons in different spatial locations 
of the network. Translationally symmetric neural fields support a continuum of stationary bump 
solutions, which may be centered at any spatial location. Random fluctuations are introduced by 
modeling the system as a spatially extended Langevin equation whose noise term we take to be 
multiplicative or additive. For nonzero noise, these bumps are shown to wander about the domain 
in a purely diffusive way. We can approximate the effective diffusion coefficient using a small noise 
expansion. Upon breaking the (continuous) translation symmetry of the system using a spatially 
heterogeneous inputs or synapses, bumps in the stochastic neural field can become temporarily 
pinned to a finite number of locations in the network, fn the case of spatially heterogeneous synaptic 
weights, as the modulation frequency of this heterogeneity increases, the effective diffusion of bumps 
in the network approaches that of the network with spatially homogeneous weights. 
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1. Introduction. Spatially localized patterns of persistent neural activity (bumps) 
are well studied phenomena thought to subserve a variety of processes in the brain [69] . 
Working (short term) memory tasks are the best known examples of brain functions 
that may exploit the fact that bumps are localized in feature or physical space [32, 14]. 
For example, in oculomotor delayed-response tasks, monkeys preserve knowledge of 
a visual cue location using prefrontal cortical neurons with elevated activity that is 
correspondingly tuned to the cue location for the duration of the delay [30, 29]. There 
has been a great deal of discussion concerning the relative role of various classes of 
prefrontal cortical neurons in maintaining persistent activity [32]. One strongly sup- 
ported claim is that slow recurrent excitation is the operant synaptic mechanism for 
preserving this localized activity during the retention period [68]. 

Experimentalists have suggested that prefrontal cortical circuitry consisting of 
local recurrent excitation and lateral inhibition may underlie the formation of the ob- 
served tuning of neurons to particular cue locations [32] . Networks with such synaptic 
architecture have long been studied as a theoretical framework for neural pattern for- 
mation, with seminal studies of spatially extended neural fields carried out by Wilson 
and Cowan [71] and Amari [1]. A distinct advantage of such networks is that they 
display bistability, where stable spatially localized bumps can coexist with a spatially 
homogeneous "off' state. Another common feature of these models is that they tend 
to be (continuously) translationally symmetric, since they are spatially continuous 
dynamical system whose symmetry is preserved under reflections and arbitrary trans- 
lations [8]. Stationary localized bump solutions arising in these models have been 
used as theoretical descriptions of tuning to visual input [4, 9], head direction [74], 
and working memory [12]. These studies demonstrate that neural field models are 
a useful tool for understanding the dynamical mechanisms necessary to sustain the 
neural substrates of a variety of sensory and motor processes. 

Since stationary bumps are an idealized description of encoding location in net- 
works representing feature space, many neural field studies have examined more deeply 
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how model modifications affect the dynamics of bump solutions [15, 8]. Many stud- 
ies have also probed the effects of persistent inputs on the dynamics of neural fields 
with feedback inhibition [1, 5, 36]. For sufficiently strong inhibition, networks can 
generate spontaneous traveling waves so activity fails to lock to stationary [5, 27, 22] 
or traveling [5, 28, 39] inputs. This can lead to breathing instabilities where the ac- 
tivity pattern oscillates regularly [27, 28]. Axonal delays can also substantially alter 
the dynamics of bumps in models with lateral inhibition, leading to multibumps [17], 
oscillatory bumps [62], and anti-pulses [43]. Multibump solutions can also be gener- 
ated by introducing synaptic connectivity that is oscillatory in space [48, 47]. Aside 
from the connectivity function, the form of the firing rate function, which converts 
local synaptic inputs to an output firing rate, can also affect the shape and stability 
of stationary bumps [33, 67]. Many studies of bumps have also explored the effect 
of auxiliary negative feedback variables like spike frequency adaptation [57, 19] or 
synaptic depression [73, 40]. Substantially strong negative feedback can lead either a 
drift instability, where the bump propagates as a traveling pulse [46, 57, 19, 73, 40], 
or a breathing instability, where the edges of the bump oscillate their position in time 
[56, 19, 20]. Recently, it was shown that an auxiliary synaptic facilitation variable 
can serve to curtail the tendency of bumps in neural fields with heterogeneous con- 
nectivity to wander [38]. Thus, there is a veritable wealth of dynamic instabilities of 
bumps that have been examined in deterministic neural fields. 

Beyond these studies, there have been several analyses of spiking neuron models of 
stationary bumps [12, 14, 44]. Spiking models have the advantage of capturing finer 
timescale dynamics, for example spike time synchrony, than those of which neural 
fields are capable. Another major difference is that spiking models are often chaotic, 
leading to dynamics that can appear random. This is much more akin to the envi- 
ronment of networks of neurons in the brain, seething with fluctuations. As a result, 
a basic behavior that has been revealed in numerical simulations of bumps in spiking 
networks is wandering of the bump's mean position [12, 14, 44]. There has been very 
limited investigation of such dynamics in neural field equations [12, 38]. Nonetheless, 
in both spiking models and neural fields with noise, the variance of the bump's posi- 
tion scales linearly with time, suggesting the position as a function of time behaves 
as a purely diffusive process [12, 14, 60, 13]. This is due in part to these system 
often being translationally symmetric [14, 44, 11]. While this symmetry allow bumps 
to be initially nucleated at any point in the network, an inherent marginal stability 
makes it so that bumps are never firmly pinned to any particular location over time 
[12, 14, 44]. Thus, bump position is fragile to noise and as well as perturbations of 
the evolution equations of the underlying dynamical system, which itself contains a 
line attractor. 

The fact that bumps in noisy models of working memory wander should be no 
surprise, in light of existing data concerning the dependence of recall error on delay 
time [70, 58]. In spite of the relatively reliable correspondence between the elevation 
of neural activity and the cue location in prefrontal cortical networks [32], there is 
inevitably some error made in reporting the original cue location [70]. Interestingly, 
the amplitude of this error scales linearly in time [58] , suggesting that it may be gen- 
erated by some underlying diffusive process. Thus, for a network to have improve 
memory storage accuracy, it should reduce the effects of this diffusion as much as 
possible. This invites the question of how networks for working memory may exploit 
dynamics that are close to line attractors to improve memory recall accuracy. Some 
computational studies have suggested that relaxing the translation symmetry of line 
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attractors by introducing multiple discrete attractors may make dynamics more re- 
silient [64, 42, 11]. However, others have viewed spatial heterogeneity in networks as 
a detriment to working memory that must be overcome [60, 38]. Therefore, to make 
the theory of bump attractors for working memory more robust, we must consider 
the effects of noise and network heterogeneity and any new phenomena they bring. 

We propose to perform an in depth analysis of the diffusion of stationary bump 
solutions in neural field equations with noise. In doing so, we wish to understand how 
parameters of the model affect the degradation of the bump's initial position. Since 
oculomotor delayed-response tasks usually require recalling the location of an object 
on a circle, this suggests using a neural field model whose spatial domain is finite and 
periodic [5, 12, 9, 67]. Thus, to accompany our analysis of stochastic neural fields, 
we will review and extend some of the results for bump existence and stability in the 
deterministic ring model [65, 4, 9] 

= ^ u(Xj t) + f w{Xi y)f{u{yj t))dyi (i.i) 

where u(x,t) is the total synaptic input to spatial location x <G [— 7r,7r] at time t. A 
variation of this model, that includes spatially varying inputs, is examined in section 
3. The integral term represents synaptic feedback from the entirety of the network 
so that the kernel w(x 7 y) encodes the strength of connections from y to x. In many 
studies of the ring model, w(x, y) = w(x — y) so the network is spatially homogeneous 
[5, 12, 34, 9, 67, 39]. In particular, using the cosine weight kernel 

w(x, y) = w{x — y) = cos(x — y). (1.2) 

makes the equation (1.1) translationally symmetric in space and amenable to explicit 
analysis. We study bump solutions that arise in the case of homogeneous synaptic 
weights extensively in section 2. However, we also study the effect of spatially hetero- 
geneous synaptic connections in section 4, so that w(x,y) = (1 + aw\{ny))w{x — y). 
Particularly, we analyze (1.1) when 

w(x, y) = (1 + a cos(ny)) cos(a; — y), (1-3) 

which provides spatially heterogeneous, yet periodic, synaptic modulation whose fre- 
quency is set by the neN. Spatial heterogeneities in the weight functions of neural 
fields have been shown to lead to multibump solutions [48, 47] and to alter travel- 
ing waves [7, 41, 16]. In section 4, we study how periodic heterogeneities affect the 
stability and evolution of bumps in the presence of noise. 

The nonlinearity / is a firing rate function which converts synaptic inputs u to a 
resulting fraction of active neurons, between zero and one by definition. In line with 
experimental observations, this is often taken to be a sigmoidal function [71, 15, 8] 

f(u) = . (1-4) 

where 7 is the gain and 9 is the threshold. We can perform much of our analysis for 
a general firing rate function /, such as the sigmoid (1.4). However, one particular 
idealization that eases mathematical analysis considers the infinite gain 7 — >• 00 limit, 
so that (1.4) becomes a Heaviside step function [1, 15, 8] 

f(u) = H(u-6) = ( J \l< e [ (1.5) 
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The Heaviside firing rate function (1.5) allows us to explicitly calculate many quanti- 
ties of interest in our study. 

As mentioned, the deterministic neural field equation (1.1) has been studied ex- 
tensively as a model of neural pattern formation [5, 12, 9, 67]. The main interest of 
this paper is to consider effects of fluctuations on stationary bump solutions of (1.1). 
In particular, we will consider the general case where noise can depend multiplica- 
tively on the state variable u. Thus, we have the following Langevin equation that 
describes a noisy neural field 

dt + s 1/2 g{U(x 1 t))dW{x 1 t), (1.6) 

where U(x, t) tracks the sum of synaptic inputs at position x G (— 7r, 7r) at time t. The 
term dW(x,t) is the increment of a spatially dependent Wiener process such that 

(dW(x, t)) = 0, (dW(x, t)dW(y, s)) = C(x - y)S(t - s)dtds, (1.7) 

so that £ determines the noise amplitude, which is weak (e <C 1). Spatial correlations 
of the noise are described by the function C{x — y), which is symmetric and depends 
on the distance between two spatial locations in the network. The function g(U), 
describing the multiplicative noise, can be specified arbitrarily in a great deal of our 
analysis. Models such as (1.6) have been recently introduced as stochastic versions of 
neural field equations [46, 37, 25, 10]. Note that we can also examine the effects of 
additive noise in the framework (1.6) by simply taking the function g(U) = 1. 

The paper is organized as follows. First, in section 2 we study the effects of 
noise on bump solutions of the ring network (1.1) with spatially homogeneous synap- 
tic weights w(x,y) = w(x — y). Since bumps have a zero eigenvalue associated with 
their linear stability, indicating marginal stability and translation symmetry, the in- 
troduction of noise in the Langevin equation (1.6) leads to their wandering as a purely 
diffusive process. We compute the effective diffusion coefficient of this process as well 
as a shift in the mean width of the bump due to the multiplicative noise. We examine 
the effects of spatially dependent inputs in section 3. Since, in this case, the network 
will no longer be (continuously) translationally symmetric, stable bumps are linearly 
stable to perturbations of their mean position. Introducing noise then leads to their 
position evolving as a mean-reverting stochastic process, rather than a purely diffu- 
sive one. However, on exponentially long time scales, the bump can escape from the 
position to which they are pinned so they move to the vicinity of another discrete 
attractor of the deterministic system. In section 4, we introduce spatially periodic 
heterogeneities into the weight function so that bumps still exist, but calculation of 
their stability reveals there is no longer a generic zero eigenvalue, once again reflect- 
ing the loss of translation symmetry. As in the case of external inputs, this leads to 
pinning of bumps to a finite number of discrete attractors in the stochastic system 
(1.6) so their position evolves as a mean-reverting process. Even though bumps can 
escape from these pinned positions, they ultimately wander with a smaller effective 
diffusion coefficient than in the spatially homogeneous network. 

2. Wandering bumps in spatially homogeneous network. We begin by 
studying bumps in a spatially homogeneous ring model (w(x,y) = w{x — y)), first in 
the absence of noise (1.1) and then with multiplicative noise (1.6). Previous studies 
of traveling fronts in reaction diffusion equations and neural fields have found multi- 
plicative noise can alter the mean speed of the front and causes the front to wander 



dU(x,t) 
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diffusively [3, 55, 63, 6, 10]. Analyzing (1.6) reveals that multiplicative noise leads to 
dynamics whose mean is given by a bump with a position that wanders diffusively. 
Our analysis allows us to approximate the diffusion coefficient of the bump, estimating 
the error a network may make in a working memory task that relies on the position 
of the bump center [70, 58, 12, 13]. 

2.1. Existence. To begin, we derive stationary bump solutions. As opposed 
to the method of construction of Amari [1], we need not presume a Heaviside firing 
rate function (1.5) to derive explicit bump solutions. We exploit the fact that the 
cosine weight function (1.2) is separable through a trigonometric identity to reduce the 
existence and stability problems to root-finding problems or linear algebraic systems 
[34, 67]. 

Upon assuming a stationary solution u(x,t) = U(x), the scalar equation (1.1) 
with a spatially homogeneous weight function w(x — y) requires that it satisfy the 
integral equation 



U{x) = f w(x- y)f(U(y))dx. (2.1) 

For the weight function (1.2), we can employ the trigonometric identity 

cos(x — y) = cos x cos y + sin x sin y, (2.2) 

so that (2.1) becomes 

U(x) = Acosx + Bsinx, (2.3) 

where 

/TT PTT 

cosxf(U(x))dx, B= smxf(U(x))dx. (2.4) 

-TT J —TT 

We look specifically for even symmetric stationary bump solutions, as is often done 
in analyses of localized solutions in neural fields [1, 15, 67, 8]. Thus, B = 0, so 

U(x) = Acosx, (2.5) 

and we can solve for A by requiring self consistency of the solution U = A cos x such 
that (2.4) becomes 

A= cosa;/(Acosx)da;. (2.6) 

J — TT 

For a general sigmoidal firing rate function (1.4), one could determine A using a 
numerical root finding method. 

For a Heaviside firing rate function (1.5), we can solve exactly for the amplitude 
A. Equation (2.5) shows U(x) is unimodal and symmetric, so it will cross above 
and below 6 at locations x — — a and x = a respectively. This provides us with the 
threshold conditions U(±a) = 9 for (2.5), which can be written equivalently as 

a^cos -1 -. (2.7) 
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Thus, we know U(x) > 9 for x G (—cos 1 {9/A),cos 1 {9/A)) 1 so the self-consistency 
condition (2.6) becomes 

A = 2 J cosrrda; = 2 sin ^cos" 1 —J = 2^ 1 - (2.8) 

Solving (2.8) for the bump scaling factor 

A= VTTo ±VT~9 (2.9) 

reveals there are two bump solutions 

U{x) = (yT+l±VT~0^ cos x, (2.10) 

and we can show that the wide solution (+) is stable and the narrow solution (— ) is 
unstable, forming a separatrix between the wide bump and the rest state U(x) = 0. 
Applying (2.7), half-widths a, can be easily computed 

a± = cos . (2.11) 



As we have mentioned, the network with a cosine weight kernel (1.2) is translationally 
symmetric, so that we could construct a bump solution centered at any position 
x G [— 7r,7r]. This would simply lead to a system of two equations for A and B 
associated with (2.3), but the width of such a bump would be the same as that of 
(2.5). We can also show this by calculating the linear stability of bumps in the network 
(1.1), revealing marginal stability of a shift perturbation. This we do now for the case 
of a general firing rate function /. 

2.2. Stability. Linear stability of bumps (2.5) can be computed by analyzing the 
evolution of small, smooth, separable perturbations such that u(x, t) — U(x)+ip(x)e xt 
for IVK 2 -)! <C 1 . Plugging this expansion into the evolution equation (1.1), Taylor 
expanding, applying (2.1), and studying first order equation yields [9, 18, 67] 

(A + l)ij(x) = r w(x - y)f'(U(y)My)dy. (2.12) 

J — 7T 

For the cosine weight function (1.2), we can apply the identity (2.2) so that 

(A + l)ip(x) = A cos x + £> sin x, (2.13) 

where 



/7T f'TT 
cosxf{U{x))ip(x)dx, B= smxf'(U(x))ip(x)dx. 

-7T J — 7T 



(2.14) 



Thus, we reduce the infinite dimensional equation (2.12) to a 2 x 2 linear spectral prob- 
lem (2.13). Such a technique was recently shown for a general class of weight functions 
in [67]. Plugging the form of tp(x) given by (2.13) into the system of equations (2.14), 
we have 

A \-f I ( cos2x ) I(cosxsinx) A / A A . . 

(A + l) I B I - I I(cosxsinx) I(sin 2 x) )\b)> {2Ab) 
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where 

l{r{x)) = f r{x)f'(U(x))dx. (2.16) 

J — 7r 

First of all, note that the essential spectrum is A = — 1 and thus does not contribute 
to any instabilities. Upon integrating (2.6) by parts, we see 

/TT />TT 
cosxf(Acosx)dx = A sin 2 xf'(Acosx)dx. (2-17) 
-TT J —TT 

Therefore, as long as A ^ 0, the equality (2.17) tells us 

Z(sin 2 x) = f sm 2 xf'(U(x))dx = 1. (2.18) 

J— TT 

Using this identity (2.18) and the fact that (2.16) is linear, we can then compute 

I(cos 2 x) = 1(1 - sin 2 x) = 1(1) - I(sin 2 x) = 1(1) - 1. (2.19) 
Finally, we can use integration by parts to show 

/TT f'TT 
cosxsmxf'(U(x))dx = — smxf(U(x))dx = 0, (2.20) 
-TT J —TT 

since U(x) is even. Using the identities (2.18), (2.19), and (2.20), it is straightforward 
to compute the eigenvalues that determine the stability of the bump (2.5). We do so 
by finding the roots of the associated characteristic equation 

A 2 + (2-Z(l))A = 0, 

which reveals the zero eigenvalue A c = 0, associated with the constant B, defined in 
(2.14), which means it reveals the linear stability of bumps in response to odd (shift- 
ing) perturbations. The fact that A is zero arises due to the underlying translation 
symmetry of (1.1) when w(x,y) is the cosine weight function (1.2). In addition, the 
stability of the bump (2.5) is determined by the sign of the other eigenvalue 

A e = 2 f f'(U(x))dx - 2, (2.21) 
Jo 

associated with A, defined by (2.14), and thus even (expanding or contracting) per- 
turbations of the bump. 

In the limit of infinite gain 7 — > 00, / becomes the Heaviside (1.5), and 

fl(TTM ) _ dH(U(x)) _ Sjx - a) S(x + a) 

in the sense of distributions, so the formula for the generically nonzero eigenvalue will 
be 

x * = - 2 + WWy (2 ' 23) 

for the bump (2.10) of half-width a. Identifying threshold 6 values at which (2.23) 
crosses zero will give the location of a saddle- node bifurcation [1, 18, 27]. Equation 
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(2.23) allows us to compute eigenvalues exactly for the wide and narrow bumps since 
(2.11) gives the half-widths and the spatial derivative at the edges 

\u'(a±)\ = (vT+0± vT 

= (VTTe±VT^e) f ± ^-°\ (2.24) 

Plugging the expression (2.24) into (2.23) yields 

A e = A± = -2+— — = -j^ (2.25) 

the nonzero eigenvalue associated with the wide (+) and narrow (— ) bump. To 
identify the threshold 9 where the two pulses annihilate in a saddle-node bifurcation, 
we look for where A± = 0. Imposing this requirement on (2.25) means 



(VTTe ± VT~e^j \Ji ± \/i - e 2 = V2. (2.26) 

It can be shown that (2.26) is equivalent to finding zeros of the quartic 6» 4 + 26> 2 - 3, 
whose real solutions are 9 = ±1. Thus, as 9 is increased from zero, the stable wide and 
unstable narrow bump branches will coalesce in a saddle- node bifurcation at 9 = 1. 

2.3. Noise-induced wandering of bumps. We now consider the effect noise 
has on bumps by studying approximate solutions to the Langevin equation (1.6) with 
a spatially homogeneous weight function w(x,y) = w(x — y). The primary behavior in 
which we are interested is how the bump's position changes. Wandering of bumps was 
first observed numerically in modeling studies of working memory that employed rate 
[12] and spiking models [14]. These authors rightly observed that such pure diffusion 
was due to the potential landscape of the deterministic dynamical system being a line 
attractor [12, 11]. In the case of truly multiplicative noise, we show that the mean 
width of the bump changes changes as well. Mainly, we show that we can use a linear 
expansion to approximate the influence of spatially correlated multiplicative noise on 
the position of bumps in a neural field. This reveals that the bump undergoes pure 
diffusion whose associated coefficient we can derive from our asymptotic analysis. 

The fact that multiplicative noise alters the mean width of the bump arises from 
the fact that this noise does not have zero mean, (g(U)dW) ^ 0. We can calculate 
this average using Novikov's theorem [54, 3, 63, 10] 

e 1/2 (g{U)dW) = eC(0)(g'(U)g(U))dt. (2.27) 

One method for deriving the result (2.27) is to Fourier transform (1.6) and evalu- 
ate averages using the corresponding Fokker-Planck equation in Fourier space [63]. 
Rewriting equation (1.6) using (2.27), we can formulate the fluctuating term so that 
it has zero mean 



dU(x,t) 
where 



/7T 
w(x-y)f(U(y,t))dy 
-7T 



dt + e 1/2 dZ(U,x,t), (2.28) 



h(U(x, t)) = U(x, t) - eC{0)g'{U{x, t))g(U(x, t)) (2.29) 
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and 

dZ(U, x, t) = g{U)dW{x, t) - s 1/2 C(0)g' (U)g{U)dt. (2.30) 

The stochastic process Z has zero mean and variance 

(dZ(U, x, t)dZ(U, y, t)) - (g(U(x, t))dW(x, t)g{U{y, t))dW{y, t)) + 0{e x l 2 ). (2.31) 

Next, we assume that the multiplicative noise in (2.28) generates two phenomena 
that occur on disparate time scales. Diffusion ol the bump from its original position 
occurs on long timescales, and fluctuations in the bump profile occur on short time 
scales [52, 3, 10]. Thus, we express the solution U of equation (2.28) as the sum of a 
fixed bump profile U £ displaced by A(t) from its mean position x, and higher order 
time-dependent fluctuations e 1 / 2 ^ + e$i + e 3 / 2 $ 2 H in the profile of the bump 

U(x, t) = U s (x - A(t)) + s 1/2 <5>{x - A(t), t) + • • • , (2.32) 

so A(t) is a stochastic variable indicating the displacement of the bump U £ with 
slightly altered half- width a £ . To a linear approximation, the stochastic variable 
A(t) undergoes pure diffusion with associated coefficient D(e) = 0(e), as we show. 
The expansion (2.32) is not a standard small-noise expansion, since the modified 
bump U £ implicitly depends upon e, where the subscript denotes parametrization. 
By substituting (2.32) into equation (2.28) and taking averages, we find the leading 
order deterministic equation for U £ is 

h(U £ (x)) = ^ w(x - y)f(U £ (y))dy. (2.33) 

The mean pulse half-width a £ and profile U £ depend non-trivially on the noise strength 
e, since h, given by (2.29), is e-dependent. Therefore, the width a £ ^ a for e > and 
ao = a, the width of the bump in the absence of multiplicative noise. Proceeding to 
next order, and requiring (2.33), we find A(t) — 0(e 1 ^ 2 ) and 

d$(x, t) = C$(x, t) + £ - 1/2 U' £ (x)dA(t) + dZ(U £ ,x, t), (2.34) 

where C is the non-self-adjoint linear operator 

C P (x) = -h'(U £ (x))p{x) + j w(x- y)f(U £ (y))p(y)dy, (2.35) 

J — TT 

for any function p(x) G L 2 [—ir,ir]. 

Upon differentiating (2.33) and integrating the convolution by parts 

h'(U £ (x))U' £ {x) - T w(x-y)f'(UM)U' £ (y)dy, 

J —TT 

so U' e (x) belongs to the nullspace of C. Now, we can ensure a bounded solution 
to equation (2.34) exists by requiring the inhomogeneous part is orthogonal to all 
elements of the nullspace of the adjoint operator C* . The adjoint is defined with 
respect to the L 2 inner product 



/7T nTT 
[Cp(x)} q(x)dx = / p(x) [C*q(x)} dx, 
-TV J —TV 
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where p(x), q(x) e L 2 [—tt,tt]. Thus, 

C*q{x) = -h'(U £ (x))q(x) + f'(U £ {x)) f w(x - y)q(y)dy. (2.36) 

There is a single function ip e (x) spanning the one-dimensional nullspace of £*, which 
we can compute explicitly for a general firing rate function /. Thus, we impose 
solvability of (2.34) by taking the inner product of both sides of the equation with 
respect to (f £ (x) yielding 



dz = 0. 



(p £ (x) U' e (x)dA(t) + e 1/2 dZ(U £ , x, t) 
Isolating dA(i), we find A(i) satisfies the stochastic differential equation (SDE) 

ip £ (x)dZ(U £ , x, t)dx 



dA(t) 



T l/2J_ I _ 



[ V <p e (x)U' e (x)da 

J —TV 



(2.37) 



With the SDE (2.37) in hand, we can compute the effective diffusivity of the bump 
to a linear approximation. First, note that the mean position of the bump averaged 
over realizations does not change in time 



(A(t)) = e 



. n il v fe{x){Z{U e ,x,t))Ax 
j\tp £ {x)U' e {x)dx 



t = 0, 



where we set the bump's initial position to be A(0) = without loss of generality. 
Computing the variance of the stochastic variable A(i), we find it evolves according 
to pure diffusion since 



(A(t) 2 ) = e 



j:j:^ £ (x)My)9(U £ (x))g(U £ (y))(W(x,t)W(y,t))dyda 



j\ V e{x)U'{x)dx 



t 



(A(t) 2 ) = D(e)t, 
and using the definition of W(x, t) in (1.7) yields 

J! v W ip £ (x)ip £ (y)g(U £ (x))g(U £ (y))C(x - y)dydx 



D{e) 



j\ip £ {x)U'{x)dx 



(2.38) 



(2.39) 



To calculate the diffusion coefficient D(e) for specific cases, we need to compute the 
constituent functions U £ (x) and (fi £ (x). We will continue to use the cosine weight 
kernel (1.2) in this analysis. 

2.4. Calculating the diffusion coefficient. First, we study additive noise, 
where g(U) = 1, performing calculations for a general firing rate function /. For 
additive noise, the modification to the fluctuating term in the Langevin equation 
(1.6) using (2.27) and (2.29) is not necessary because (g'(U)g(U)) = so h(U) = U. 
Also, we find that equation (2.33) becomes (2.1), so that 



U' £ (x) = U'(x) 



-A sin: 



(2.40) 
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where A is defined by (2.6). Along these lines, the nullspace <p e (x) of the adjoint C* 
will depend trivially on e as well so f s (x) = f(x). To find ip(x) in the case of additive 
noise, we write the (2.36) using h(U) = U and the kernel w(x) — cosx so 

<p(x)=f'(U(x)) f cos(x-y)<p(y)dy. (2.41) 

J — TT 

Using separability of the cosine kernel (2.2), we find ip must satisfy 

ip{x) = Cf'(U(x))cosx + Sf'(U(x))smx, (2.42) 

where 



/TT /> TT 

cosx(p(x)dx, S= sinx(p(x)dx. 

-TT J —TT 



(2.43) 



Plugging the expression (2.42) into the pair of equations (2.43) gives us the linear 
system 

C = I(cos 2 x)C +I(cosxsinx)S', 

5=I(cosxsina;)C+X(sin 2 x)S*, (2.44) 

where l(r(x)) is given by (2.16). Upon applying the identities (2.18), (2.19), and 
(2.20), the system (2.44) becomes 

2C=X(1)C, S = S. 

As the gain 7 and threshold 9 of the sigmoid (1.4) are varied 1(1) 7^ 2 almost ev- 
erywhere in (7,0). Thus, the only non-trivial solution to (2.41) consistent across 
parameter values requires C — 0, so 

<p(x) = f (U(x))smx, (2.45) 

up to the scaling S. Thus, for a general sigmoid (1.4), we can use our formula for the 
spatial derivative (2.40) along with (2.45) to compute the term in the denominator of 
the diffusion coefficient (2.39) given 

<p(x)U'(x)dx = -A sin 2 xf'(U(x))dx = -A, 

-TT J —TT 

applying (2.18). Thus, the effective diffusion coefficient in the case of additive noise 
is given 



D(e) = J5 f f sinxsmy f'(U(x))f'(U(y))C(x - y)dydx. (2. 



46) 



To determine the diffusion coefficient (2.46), we must specify correlation function 
C(x — y). Two limits, spatially homogeneous and spatially uncorrelated noise, will 
help us understand how the spatial profile of the noise affects diffusion. In the limit 
of spatially homogeneous correlations (C(x — y) = Co), the neural field specified by 
(1.6) is driven by a spatially homogeneous Wiener process dWo(£). In this case, the 
bump will not diffuse at all since (2.46) simplifies to 



D(e) - W 



smxf'(U(x))dx 



0, 
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Fig. 2.1. Wandering of bumps due to additive noise with cosine correlation function (2.47), 
such that g(U) = 1, in ring model (1.6) with Heaviside firing rate function (1.5) and cosine weight 
kernel (1.2). (a) Single realization of neural activity U {x , i) driven by additive noise with amplitude 
e = 0.001, using stable stationary bump (2.10) as initial condition. Superimposed line tracks center 
position (peak) of bump. Threshold 9 = 0.5. (b) Bump wanders more for higher amplitude noise 
e = 0.01. (c) Variance (A(t) 2 ) of bump center position computed across 1000 realizations (red 
dashed) scales linearly with time, as predicted by theory (blue solid). Diffusion coefficient D(e) 
computed using (2.49). Parameters 9 = 0.5 and e = 0.01. (d) Dependence of diffusion coefficient 
of network threshold 9 for e = 0.001 and e = 0.01 computed using asymptotic approximation (2.49) 
(blue line) and computed numerically (red circles) across 1000 realizations run for 50 time units. 
Numerical simulations of (1.6) are performed using Euler-Maruyama with a trapezoidal rule for the 
integral with the discretization Ax = 0.01 and At = 0.01. 

since f'(U(x)) is even. Only the width of the bump will fluctuate, which is not tracked 
by our first order approximation. 

In the limit of no spatial correlations (C(x — y) — > S(x — y)), every spatial point 
receives noise from an identically distributed independent Wiener process. 1 In this 
case, we can simplify (2.46) to find 

D{e) = J k f sin Mm*))] 2 d*, 

which is nonzero for e > 0. 

Now, to compare our asymptotic analysis to numerical simulations, we will study 



1 One important fact to note is that if we attempt to numerically simulate (1.6) with spatially 
uncorrelated noise on a spatial mesh of width Ax, a nonzero correlation length Ax arises from the 
discretization [3, 8]. 
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the effect of a cosine spatial correlation function 

C(x - y) = 7r cos(x - y). (2.47) 

The 7r scaling factor arises when we compute the correlation function from the spatial 
noise filter given by an unsealed cosine T(x) — cosx. To see this we take a spatially 
uncorrelated Wiener process dT(x,t) and filter it with a cosine to give 

T(x - y)dT(y, t)dy = / cos(a; - y)dT(y, t)dy, 

-7T J — 7T 

where (dT(x,t)) = and (dT(x, t)dT(y, s)) = S(x — y)S(t — s)dtds. Then the cross 
correlation of dW(x,t) is given 



(dW(x,t)dW(y,s)) = / cos(a; - x 1 ) cos(y - y')(dT(x', t)dT(y', s))dx'dy' 

J — TV J —7T 

/TV 
cos(x — x') cos(y — x')dx'5(t — s)dtds 

{dW(x, t)dW{y, s)) = C(x - y)S(t - s)dtds = tt cos(x - y)S(t - s)dtds, 

as given by (2.47). Therefore, in the case of cosine spatial correlations (2.47), the 
diffusion coefficient in the presence of additive noise (2.46) becomes 

D(e) = % f f smxsmyf(U(x))f'(U(y))co S (x-y)dydx (2.48) 
sin 2 xf'(U(x))dx^j + (^J sin x cosx j '' '({/(x))dx^ 



£7T 

A 2 



£7T 
A 2 ' 



where we have applied the identities (2.2), (2.18), and (2.20). In the case of a Heavisidc 
firing rate function (1.5), we can use the explicit expression (2.9) for the amplitude of 
the stable bump to write (2.48) simply in terms of the noise amplitude e and network 
threshold 9 as 

D{e) = £n (2.49) 

W 2 + 2\/r^ 2 v ; 

Thus, we have an asymptotic approximation for the effective diffusion coefficient D(e) 
of a stable bump (2.10) in the ring network (1.6) driven by additive noise, giU) = 1. 
We compare (2.49) to diffusion coefficients computed from numerical simulations in 
Fig. 2.1. As predicted by our theory, averaging across numerical realization the 
Langevin equation (1.6) shows the variance of the bump's position scales linearly in 
time. 

Now, we examine effects of multiplicative noise. The main difference between this 
case and that of purely additive noise is that the mean width and amplitude of the 
bump are altered, as suggested by (2.33). In the case of a cosine weight kernel (1.2) 
and a linear multiplicative function giU) = U, the modified equation (2.33) for the 
bump becomes 

i r 

u £ {x) = ^ _ £ c(o)) J cos yf( u e(y)) d y cosx = Acosx. (2. 50) 
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The final equality of (2.50) shows that the form of the bump solution is the same as 
in noise-free case (2.5); only the amplitude is changed. Equation (2.50) can then give 
us a nonlinear equation specifying the amplitude A £ according to 

i r 

Ae = (l - " eCW) J cosx /(^ cosx ) dx - ( 2 - 51 ) 

As in the noise-free case, this equation is much easier to solve than the nonlinear 
integral equation (2.50). This also provides the spatial derivative for the mean bump 
profile 

U' e (x) = -A £ sinx. (2.52) 

Notice, by integrating the right hand side of (2.51) by parts and canceling A £ , as long 
as A E ^ 0, we have the formula 

l-sC(0)= f sin 2 xf'(U £ (x)dx =I £ (sin 2 x), (2.53) 

J — 7f 

where 

l £ (r(x)) = f r(x)f'(U £ (x))dx. (2.54) 

J — 7T 

Notice, (2.53) is analogous to the identity (2.18). As in that case, we can derive 

X £ (cos 2 x) =1(1) +eC(0) - 1, I £ (cosxsinx) = 0. (2.55) 

With the modified bump solution (2.50) in hand, the ^-dependent nullspace <p £ (x) 
of the adjoint operator (2.36) is given by 

(1 - eC(0))<p e (x) = f'(U £ (x)) f cos(x - y)<p e {y)d.y. (2.56) 

J — 7r 

Following our analysis in the case of additive noise, we use the identity (2.2) of the 
cosine kernel (1.2) to derive a linear system specifying (p £ (x) and apply the identities 
(2.53) and (2.55) to yield the solution to (2.56) given 

V>M = f'(U e (x))smx. (2.57) 

We then can use the spatial derivative (2.52) and the nullspace formula (2.57) to 
compute the term in the denominator of the diffusion coefficient (2.39) given by 

f <p £ (x)U' £ (x)dx = -A £ f sm 2 xf'(U £ (x))dx = -A £ (l-eC(0)), 

where we have applied (2.53). Thus, the effective diffusion coefficient in the case of 
multiplicative noise with g{U) = U becomes 

D{e) = £ ^Q^ 2 J J cosxsmxf'(U £ (x))coaysmyf'(U £ (y))C(x-y)dydx. 

(2.58) 

As we found in the case of additive noise, when noise is spatially homogeneous 
(C(x) = Co), the diffusion coefficient defined by (2.58) is D(e) = 0. However, for 
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Fig. 2.2. Wandering of bumps induced by multiplicative noise, such that g(U) = li, in ring 
model (1.6) with Heaviside firing rate function (1.5) and cosine weight kernel (1.2). (a) Single 
realization of U(x,t) driven by multiplicative noise with amplitude e = 0.01, using stable station- 
ary bump (2.10) as initial condition. Superimposed line tracks center (peak) of bump. Threshold 
8 = 0.5. (b) Dependence of diffusion coefficient of network threshold 9 computed using asymptotic 
approximation (2.49) (blue solid) and computed numerically (red circles) across 1000 realizations 
for 100 time units. Numerical scheme is same as in Fig. 2.1. 



e > 0, the mean profile of the bump is modified, due to the C(0) = Co dependence of 
the modified bump amplitude equation (2.51). We cannot study the case of spatially 
uncorrelated noise ((C(x) — > S(x))) analytically, due to the delta distribution singu- 
larity (C(0) — > oo ) appearing in our explicit formulae for the amplitude A £ (2.51) and 
diffusion coefficient D(e) (2.58). Thus, we proceed to analyze the case of spatially 
structured correlations. 

In the case a cosine profile of spatial correlations (2.47), we can use the identity 
(2.2) to find that (2.58) becomes 

D{e)= (y sm 2 xcosx f\U e {x))Ax^j , (2.59) 

where a second integral term vanishes, due to its integrand being odd. In the case of 
a Heaviside firing rate function (1.5), we can compute the diffusion coefficient (2.58) 
explicitly. To start with, we find the nonlinear equation (2.51) for the amplitude of 
the modified bump solution 



A _ y/l + 0(1 - 7Tg) + ^1 - 0(1 - re) 

1 — 7T£ 

This implies that the modified bump half-width a = cos~ 1 {9/A e ) is 

-i ( v 1 + (i - - V 1 - a - (0 m s 

a e = cos I — y - I . (2.60) 

Now, we can compute the diffusion coefficient (2.59) explicitly 

eir ( sin 2 a e cos a e \ 2 2 eir(l - Trs) 2 9 2 

Die) = -7T — : = £7rcos a £ = . (2.61) 

(1-£tt) 2 V A £ sma £ J 2 + 2y/l - (1 - Tre) 2 8 2 
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Fig. 2.3. Extinction of bumps in the network (1.6) with Heaviside firing rate function (1.5) 
and additive noise with cosine spatial correlations (2.47). (a) Numerical simulation of (1.6) in 
presence of additive noise (g(U) = 1), with threshold 8 = 0.95 and noise amplitude e = 0.01, where 
noise causes bump extinction at t fa 65. (b) Numerical approximations (red circles) to the mean 
bump extinction time T ex tinct across 1000 realizations, given by when the bump's peak crosses below 
threshold 8. This is fit to the exponential function 6exp(7|0 — 0$n\) of the distance to the saddle- 
node at 6 = 8g?f using least squares (blue line). Specifically, b fa 10 and 7 fa 33. Noise amplitude is 
e = 0.01. Numerical scheme is the same as in Fig. 2.1. 

We compare our asymptotic estimation of the effective diffusion coefficient (2.61) to 
the results of numerical simulations in Fig. 2.2. Notice the different scaling of the 
diffusion coefficient as compared to that in the case of additive noise (2.49), especially 
in the vicinity of 9 m 0. This is due to the fact that lower network thresholds lead 
to there being weaker noise near the edges of the bump in the case of multiplicative 
noise. 

2.5. Extinction of bumps near a saddle-node. In general, there are few 
analyses that approximate the waiting times of large deviations in spatially extended 
systems with noise [24, 66]. Recently, the approach of calculating minimum energy of 
the potential landscape of such systems has been used as a means of approximating 
the path of least action, along which a rare event is most likely to occur [59] . Here, 
we show an example of a large deviation in the stochastic neural field (1.6) where the 
dynamics escapes from the basin of attraction of the stationary bump solution (2.5). 

We find that noise can cause trajectories of U(x,t) to cross through a separatrix 
of the deterministic system (1.1). This unstable manifold separates stable bump 
solutions from the homogeneous "off" state. When multiplicative noise has no additive 
part (.g(0) = 0), we expect the U = state to be absorbing. In Fig. 2.3(a), we show 
the results of simulations where g(U) — 1, so that noise is purely additive. Here we 
take a Heaviside firing rate function (1.5) and the threshold 9 = 0.95, so the system 
is operating near the saddle-node bifurcation of the deterministic system at 9sn = 1 
(see equation (2.26)), and additive noise causes the bump to temporarily wander and 
then extinguish. Relating this to oculomotor delayed-response tasks, such an event 
would cause major error in the recall of a cue location. In Fig. 2.3(b), we show 
the mean time to extinction T ext i nc t depends exponentially on the distance of the 
system to the saddle-node bifurcation, as described by the function bexp(j\9 — 9sn\)- 
While we do not have a derivation of this formula per se, it stands to reason that 
the dynamics escapes some potential well whose height can be characterized by the 
distance \9 — 9$n\- Thus, a Kramer's escape rate calculation could give the desired 
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result [31]. We will leave such analysis to future studies. 

3. Locking and escape of input-driven bumps. Several studies of the ring 
model (1.1) have considered it to be an idealized model for the visual processing of 
oriented inputs [34, 5, 9, 67]. However, none of these have examined how well networks 
represent stimuli when they receive some source of noise. Here, we will study how 
well a network locks to a stationary stimulus, in the presence of purely additive noise 



dU(x,t) 



U(x, t) + w(x- y)f(U(y, t))dy + I(x) 



dt + e 1/2 dW(x,t). (3.1) 



In the context of networks the encode working memories, an external input could be 
interpreted as feedback projections from another participating layer of neurons that 
may mirror the storage of (3.1). Note, we could carry out an analogous analysis in 
the presence of multiplicative noise, but the formulation (3.1) makes effects of input 
and noise more transparent. The term I{x) represents a persistent external input to 
a network. For example, including 

I(x) = I a cosnx (3.2) 

as our input turns the energy landscape of the deterministic ring network (1.1) from a 
line attractor (with continuous translation symmetry) to a chain of multiple attractors, 
such that the network now has dihedral D n rather than circular 0(2) symmetry. Note, 
any break in continuous translation symmetry, however weak, will considerably alter 
the governing dynamics of the deterministic system (1.1). In particular inputs can 
pin bumps in place so they do not wander freely. 

3.1. Existence of input-driven bumps. Considering the deterministic version 
of the input-driven network (3.1) with the n-modal input (3.2), we first examine the 
stimulus driven bump solutions to the system 

du(x t) r 

V m ' =-u(x,t)+ / w(x-y)f(u(y,t))dy + I Q cos(nx). (3.3) 



Looking for symmetric stationary solutions u(x,t) — U(x), in the case of cosine 
weights (1.2), the equation (3.3) becomes 

U(x) = / cosy f(U(y))dy cos x + I cos(nx). (3.4) 

J — TT 

Note, there is another class of solutions centered at x = n/n, rather than x = 0. Since 
these solutions are always unstable for the ranges of parameters we are examining, 
we will ignore them for the time being. We will study such solutions in more detail 
in section 4, when we study a related system that considers spatial heterogeneity in 
synapses, rather than input. Thus, the solutions we study here will be of the form 

U(x) = Ax cosx + I cos(nx), (3.5) 

and we can write down an implicit equation for A\ by requiring self consistency of 
the solution (3.5), so the amplitude is given 



-f 



A\ = / cosy f (Ax cosy + J cos(ny))dy, (3.6) 
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Fig. 3.1. Input-locked bumps in the deterministic neural field (3.3). (a) Bump half-width a for 
a unimodal (n = I) and bimodal (n = 2) stimulus of form (3.2) calculated using (3.9) and (3.11), 
demonstrating the dependence of the stable (black) and unstable (grey) branches on the input strength 
Iq. (b) Stable (black) and unstable (grey) bump solutions in the case of a bimodal stimulus (dashed) 
of strength Iq = 0.4. Threshold is fixed at 8 = 0.5. 



and in the special case, n = 1, we have 

Ai = f cosyf((A 1 +I )cosy)dy. (3.7) 

J —ft 

To demonstrate this analysis, we consider the case of a Heaviside firing rate function 
(1.5). It is straightforward to evaluate the integral (3.6) using the fact that U(x) is 
unimodal and thus only superthreshold in the region x e (—a, a), due to symmetry, 
so 

,a 

Ai = I cosydy = 2 sin a, 

J — a 

then prescribing the threshold equation for self consistency yields 

2 sin a cos a + Iq cos na = sin 2a + Io cos na = 9, (3.8) 

which, in general, we can solve using a numerical root finding algorithm for the bump 
half-width a. Of course, as n is increased, there are higher frequency modulations of 
the input (3.2) that affect the profile of the bump. This can create problems in the 
requirement that the superthreshold region be a connected domain, that is that 

U(x) > 9 : \x\ < a and U(x) < 9 : \x\ > a. 

Essentially, we need to guard against multibump solutions arising [48, 47], as this 
complicates our analysis. In light of this, we restrict our study to small values of n 
and Jo- 

To start, we note that, in the special case n = 1, our equation for the bump 
half-width (3.8) becomes 

(2 sin a + 7 ) cos a = 9. (3.9) 

Because of the unimodality of the input I{x) = Iq cos x, we do not need to impose any 
additional super or subthreshold conditions, since the spatial frequency of the input 
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(n = 1) is commensurate with the spatial frequency of the bump. We demonstrate 
the dependence of the bump half- width a upon the input strength I in Fig. 3.1(a). 
More interesting, however, is the implicit equation for the case n = 2, given 

sin 2a + i cos 2a = 9. (3.10) 

The equation (3.10) is explicitly solvable for the half- width a in terms of parameters 
9 and Io- To show this, we first multiply by (i + 0) and apply half-angle formulae to 
yield 

(io + io#)(cos 2 a - sin 2 a) + 2(i + 9) cosasina = 9 2 + I 6. 
Upon rearranging terms, we can formulate the system as the equivalence of two squares 

(1 — 9 2 + ip) cos 2 a = cos 2 a — 2(I + 9) cosasina + (io + 9) 2 sin 2 a 
whose square roots can then be taken to yield 

±\J 1 — 9 2 + Iq cos a = cos a — (I + 9) sin a. 

Then, upon rearranging terms, we can write the problem in terms of the level set 
crossings of the tangent function 



1 ± y/1 - 9 2 + I 2 

tana= —9 ' 

so that we can explicitly express the half-width a of the bump in terms of parameters 



a = tan 1 



1 ± y/1 - g 2 + I 2 

i + e 



(3.11) 



where we restrict the range of tan -1 to yield a e [0,7r]. We demonstrate the depen- 
dence of the half- width a on the input strength Iq in Fig. 3.1(a). In addition, we 
show how the profile is altered by a bimodal input in Fig. 3.1(b). Now we turn to 
analyzing how inputs alter the stability of stationary bumps in the network. 

3.2. Stability of bumps locked to inputs. As has been shown previous stud- 
ies, stationary inputs can produce bumps that are linearly stable to translating pertur- 
bations, even though the input-free system is marginally stable to such perturbations 
[5, 9, 27, 67]. Here, we demonstrate these results, as they provide intuition as to the 
alteration of the stochastic dynamics of the system (3.1) from the input-free system 
(1.6). To do so, we perturb about U(x) with small, smooth, separable functions us- 
ing u(x,t) = U(x) +ip(x)e xt , where \ip(x)\ "C 1, and perform a regular perturbation 
expansion, studying the first order equation 

(A + l)V(z) = f w(x - y)f'(U(y))iP(y)dy, (3.12) 

J —ft 

same as the input-free case (2.12) up to the different form of U(x) given by (3.5). 
In the case of a cosine weight function (1.2), we can separate the kernel w in (3.12), 
indicating that the function ip(x) will be of form (2.13). Therefore, as in the input 
free system, we can reduce the problem to a 2 x 2 linear system given by (2.15). The 



20 



main difference here is that the bump solution is given by (3.5), as modified by the 
input. Therefore, upon calculating the eigenvalues associated with odd perturbations 
to the bump, we have 

/TT 
sin 2 yf(Ai cosy + I cos(ny))dy. (3.13) 
-TT 

Upon integrating the right hand side of equation (3.6) by parts, we find that 

/TT 77, X 

sm 2 yf'(A 1 cosy + I cos(ny))dy = l--^-j smysm(ny)f'(U(y))dy. (3.14) 

Therefore, using our equation for the eigenvalue A Q (3.13), the stability of the bump 
(3.5) to odd perturbations will be determined by the sign of 

A Q = — ^^I(sinxsin(nx)), (3.15) 
A\ 

where l(r(x)) is given by (2.16). We are particularly interested in how the input 
(3.2) alters the stability of of the bump to odd perturbations because the eigenvalue 
A Q — > in the limit Io — > 0. Therefore, infinitesimal changes in io can alter the linear 
stability of the bump with respect to these perturbations. Using the formula (3.14), 
we can also reduce the formula for the other eigenvalue 

A e = -1+ / cos 2 yf'(A 1 cosy + I cos(ny))dy, 

J —IT 

associated with the even perturbations of the bump. This becomes 

A e = -2 + 1(1) + ^X(sin x sin(nar)), (3.16) 
A\ 

whose sign will determine even perturbation stability. 

To employ the linear stability theory we have developed, we study the case of a 
Heaviside firing rate function (1.5). In this case, we know A\ — 2sina and we can 
compute the integrals so that the eigenvalue formulae (3.15) and (3.16) reduce to 



nlo sin(na) 
2 sin 2 a + Ion sin(na) 

and 

2 cos(2a) — nlo sin(na) 



A.. (3.17) 



A R 



2 sin a + Ion sin(na) 



Studying specific cases will help us understand how the input changes the stability of 
the bump (3.5). In particular, if we start with the n = 1 case, we have 

A» = -^-<0, 
2 sin a + io 

since a G [0, ir] by definition. Thus, an arbitrarily weak input will pin the bump (3.5) 
to the position x = so that it is linearly stable to odd perturbations. The eigenvalue 
associated with even perturbations will be 

2 cos(2a) — Io sin a 

= 9 1 

2 sin a + I sin a 
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whose sign is, in general, preserved from the input- free case (7 = 0). Moving to the 
n = 2 case, the odd eigenvalue will be 

_ J sin(2a) 

sin 2 a + Iq sin(2a) ' 

so that A_ < for sure when a e [0,7r/2]. The eigenvalue associated with even 
perturbations will be 

cos(2a) — Jo sin(2a) 
sin 2 a + Iq sin(2a) 

In our analysis of the stochastic network (3.1) with input (3.2), we find the linear 
stabilization of odd perturbations to the bump allows it to remain pinned to a position, 
determined by the bump's center in (3.3). This in contrast to the input-free system 
(Jo = 0), in which the bump diffuses freely in the presence of noise. 

3.3. Mean-reverting stochastic process for bump location. Now, we con- 
sider the effect of additive noise on the position of the bump in the stimulus-driven 
network (3.1) with input (3.2). Since the translation symmetry of the network has 
been broken, we find the stochastic variable describing bump location evolves as a 
mean-reverting (Ornstein-Uhlenbeck) process on moderate time scales. On very long 
timescales, large deviations occur, where the bump can escape from the vicinity of 
the stimulus peak at which it originally resided. 

To begin, we carry out a similar analysis to that of bumps evolving in the input 
free network (1.6). We express solutions to (3.1) as a combination of the bump profile 
U (same as the deterministic system) displaced by A(t) from its mean position, and 

an expansion of higher order time-dependent fluctuations e 1 / 2 Q + e$i + e 3//2 <I> 2 H 

in the shape of the bump's profile so 

U{x, t) = U(x - A(t)) + e 1/2 $(x - A(t), *) + •••. (3.18) 

Substituting the expansion (3.18) into (3.1) and taking averages, we find the leading 
order deterministic equation (3.4), giving us the input-driven bump solution (3.5). 
Proceeding to next order, we find A(t) = 0(e 1 ^ 2 ) and 

d$(.x, t) = C${x, t)dt + s- 1/2 U'(x)dA(t) + dW(x, t) + e- 1/2 I , (x)A(t)dt, (3.19) 
where C is the non-self-adjoint operator 

C P (x) = -p(x) + f w(x - y)f'(U(y))p(y)dy, (3.20) 

for any function p(x) G L 2 [—tt,tt]. Notice the last term on the right hand side of 
(3.19) arises due to the input. Since U and $ are functions of x — A(t), we have made 
the approximation I{x) = I(x - A(t) + A(t)) w I(x - A(t)) +I'(x- A(t))A(t). Now, 
we can ensure that a bounded solution exists by requiring the inhomogeneous part of 
(3.19) is orthogonal to the nullspace ip(x) of the adjoint operator C* defined by 



C*q(x) = -q(x) + f'(U(x)) f w(x - y)q(y)dy. 



(3.21) 
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Fig. 3.2. Bumps pinned by stationary inputs (3.2) in the stochastic neural field (3.1) with 
cosine correlated noise (2.47). (a) Numerical simulation for unimodal inputs (n = I). Bump 
stays in the vicinity of the stable fixed point at x = 0. (b) Variance of the bump's position computed 
across 1000 realizations (red dashed) saturates, rather than growing linearly. Theoretical curve (blue 
solid), given by Ornstein- Uhlenbeck calculation (3.26) with (3.27), compares nicely, (c) Numerical 
simulation for bimodal inputs (n = 2). Bump is initiated and stays in the vicinity of the fixed point 
at x = 0, although there is another equilibrium of the deterministic system (3.3) at x = n. (d) 
Variance of the bump's position for n = 2. Other parameters are 6 = 0.5 and e = 0.01. 

Upon taking the L 2 inner product of both sides of (3.19) with <p(x) then provides a 
sufficient solvability condition 

<p(x) \u'{x)dA(t) + 7'(i)A(i)d* + e 1/2 dW(x, t) dx = 0. (3.22) 

We can rewrite (3.22) to find that A(t) satisfies the SDE 

dA(i) + K A(t)dt = dW(t), (3.23) 

where 



j:^(x)U'(x)dx 



(3.24) 



and 



, ,„ [ v „<p(x)W(x,t)dx 
\_^ip{x)U'{x)dx 
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Note that the white noise term (3.25) has the same diffusion coefficient as we computed 
in the input-free case for additive noise, 

(dW(t)) = 0, (dW(t)dW(t)) = D(e)dt 

where D(e) is given by (2.39) with g(U) = 1. Under the assumption that we begin 
the bump at a stable fixed point, we can calculate the mean and variance of the 
Ornstein-Uhlenbeck process using standard techniques [31] 

(A(i)) = 0, (A(tf) (A(t)f = ^ [1 - c- 2Kt ] • (3.26) 

Thus, as opposed to the case of the freely diffusing bump, whose position's variance 
scales linearly with time as (2.38), the stimulus-pinned bump's variance saturates 
at D(s)/2k in the large t limit, according to (3.26). Variance saturation of bump 
attractors in networks with inputs has been demonstrated previously in simulations 
of spiking networks [72]. Here, we have analytically demonstrated the mechanism by 
which this can occur in a neural field. 

In the case of a Heaviside firing rate function (1.5), cosine synaptic weight (1.2), 
and cosine input (3.2), we have that the diffusion coefficient D(e) will be given by the 
formula (2.49) and the mean reversion rate (3.24) will be given by 

k= f^< na ) (3.27) 
2 sin a + nl sin(na) 

Not surprisingly, up to a scaling factor, this is the same as the eigenvalue (3.17) 
associated with linear stability of odd perturbations to the bump in the deterministic 
system. With the formula for k in hand, we can approximate the variance of the 
stochastic process A(t) by the formula (3.26). We compare this theory to an average 
across realizations in Fig. 3.2 for the cases n = 1 and n = 2, showing it captures the 
saturating nature of the variance. 

3.4. Noise-induced switching between two attractors. On substantially 
long waiting times, we would not necessarily expect A(t) to stay close to a fixed 
point of the deterministic system, even though we have made this assumption in our 
perturbation analysis. The bump will eventually escape to a neighboring fixed point 
(see Fig, 3.3(a)). Analogous to this, studies of mutually inhibitory neural networks 
have shown that including additive noise can cause transitions between two winner- 
take-all states of a network [53]. To our knowledge, this is the first study to examine 
such phenomena in the context of a spatially extended neural field equation. However, 
there have been a studies of the switching times between wave propagation directions 
in a neural field with local adaptation that employed numerically derived forms of an 
effective potential [46, 45]. 

We find that additive noise causes trajectories oiU(x, t) to cross through a sepa- 
ratrix of the deterministic system. Similar to our study of extinction in the input-free 
network, this separatrix is an unstable bump. Rather than separating a stable bump 
from a homogeneous "off" state, here it separates two stable bumps from one another, 
centered at x — and x — tt. In Fig. 3.3(a), we show one such transition. In this 
case, our approximation using an Ornstein-Uhlenbeck process (3.23) will clearly break 
down, since the bump is now attracted to a completely different stable state. In Fig. 
3.3(b), we show the the mean time until a switch T SW i tc h depends exponentially on the 
strength of the input Jq, given 6exp(7/ ). Essentially, we are controlling the depth 
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Fig. 3.3. Escape of a pinned bump solution from the vicinity of one stable equilibrium to 
another, (a) Numerical simulation of the stochastic neural field (1.6) in the case I(x) = 7o cos 2x. 
After a waiting time, the bump hops from x fa to x Fa 7r, the two stable fixed points of the underlying 
deterministic system, (b ) Mean waiting time to a switch as a function of the strength of the input Iq 
to the network as computed using numerical simulations (red circles). This is fit using least squares 
to an exponential bexp(yIo) (blue solid) where b = 750 and 7 = 30. Other parameters are 8 = 0.5 
and e = 0.01. 



of a bistable potential well in which the dynamics of the bump's position will evolve. 
The stronger the input, the deeper the well will be. As in the case of bump extinc- 
tion, we might expect a Kramer's escape rate calculation could give us such a result 
[61, 31]. However, we will leave such calculations to future studies of rare events in 
neural fields. 

4. Pinning and reduced diffusion due to synaptic heterogeneity. Synap- 
tic connectivity that is patchy and periodic has been identified in anatomical studies 
of prefrontal cortex [49] and visual cortex [2] using fluorescent tracers. Motivated 
by these findings, several mathematical analyses of stationary bumps in neural fields 
have employed weight kernels with periodic spatial heterogeneities [48, 47]. They 
found that such heterogeneities can lead to multiple bump solutions, where several 
disjoint subdomains of the network are active. In addition, some studies have exam- 
ined the effects that synaptic weight heterogeneities have upon the propagation of 
traveling waves [7, 41, 16], showing they can slow waves or even cause failure. 

We explore the effect synaptic heterogeneities have on the diffusion of bumps. 
Noise causes bumps to wander freely in the translationally symmetric network, so 
the memory of the initial condition deteriorates over time. However, previous studies 
of bumps in spiking networks with some spatially dependent heterogeneity in model 
parameters have shown the bump will become pinned to a few discrete positions in 
the network [74, 60]. Such symmetry breaks in the synaptic landscape of a network 
could originate from Hebbian plasticity reinforcing regions that have received more 
input during, for example, short term memory task training [23]. Mathematically, 
this can be understood as the dynamic landscape of the network switching from a 
line attractor to a chain of discrete attractors, just as we found in the input-driven 
network (3.1). Here, we study a periodic heterogeneity in synaptic weights, which 
allows us to predict the most likely position for bumps to be. Interestingly, as the 
frequency of this heterogeneity is increased, so too does the effective diffusion of the 
bump. 
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4.1. Existence of bumps. We first show that the network (1.1) with a modified 
weight kernel (1.3) supports stationary bump solutions. While multibump solutions 
may arise in networks with certain periodic synaptic heterogeneities [48], only single 
bumps arise in the network with the weight kernel (1.3). This can be easily shown by 
applying the identity (2.2) to the stationary solution problem {u(x,t) = U{x)). We 
will show that there are 2n locations x — rmr/n (to e {— n, ...,n— 1}) at which bumps 
can reside, rather than a continuum (centered at x e [— it, it]), as in the network with 
a translationally symmetric kernel like (1.2). 

To start, we show the breakdown in the translation symmetry of stationary bump 
solutions in a network with general firing rate function /. Looking for stationary 
solutions u(x,t) — U{x) 1 we find that (1.1) with the periodic heterogeneous weight 
kernel (1.3) becomes 



where w\ {x) — wi (x+2ir) . Applying an arbitrary translation b to the spatial argument 
of the bump, we find 



Thus, if a > 0, then we must restrict b = 2inn/n for any to € Z, so that w\{nz — nb) = 
w\{nz — 2nm) — w\{z). Therefore, if U(x) is a stationary bump solution to (1.1) with 
weight (1.3), then U(x + 2-Km/n), with m £ Z, is also a solution. Note also that 
reflection symmetry is preserved since 



where we have used the facts that w\ and w are even functions. 

Not only will bumps defined by (4.1) exist in the network (1.1) with heterogeneous 
weight (1.3), there will also be bumps centered at x = (2m+l)ir/n, m e Z. However, 
these will have different amplitude than those centered at x = 2rmr/n. Upon writing 



p7T 

u ( x ) = (! + vwi {ny))w{x - y)f(U {y))dy 



(4.1) 



U(x + b) 





U(x + n/n) = 




— 7T 



(1 + (jw 1 {ny)w{x - y)f(U(y + ir/n))dy, 



a change of variables yields 




— 7T 



(1 + aw\{nz — tt))w(x + ir/n — z)f(U(z))dz, 



(4.2) 



so that for the weight function (1.3), equation (4.2) will become 




— 7T 



(1 - crwi(m/))w(iE + n/n - y)f(U(y))dy, 
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Fig. 4.1. A finite number (In) of bump locations in the network (1.1) having heterogeneous 
synaptic connectivity (1.3) with modulation frequency n . (a) Bump center locations along x £ (— tt,tt] 
for various values of n have an alternating pattern of locations with a stable bump (blue filled) and 
only unstable bumps (red circles). This creates a dynamic landscape of alternating stable nodes and 
saddles in space, (b) Associated bumps determined by implicit equation (4-9) when n = 3. Stable 
bumps with amplitude A+ (4-3) centered at x = 0, i^p (blue solid). Unstable bump with amplitude 
A- (4-4) centered at x = tt, ±^ (red dashed). There are six other unstable bumps (not shown) that 
accompany each displayed bump. Other parameters are 8 = 0.5 and a = 0.2. Firing rate function is 
Heaviside (1.5). 



where we have used the fact that W\{nx — tt) = cos(nx — tt) = — cos(na;). Using 
the same arguments as for bumps centered at x = 2irm/n, there will also be bumps 
centered at x = (2m + l)ir/n. Therefore, there will be 2n total bump locations in the 
network. We can use equation (4.1) along with the weight function (1.3) to provide 
an amplitude equation for the bump U = A + cos x centered at x — 



A + = I cosx(l + <7Cos(nx))f(U(x))dx. (4.3) 

J — TT 

Similarly, the bump U = A- cos(a; + ir/n), centered at x — Tt/n, will have amplitude 
A_= [ cos(l-acos(nx))f{U(x))dx. (4.4) 



— 7T 



We demonstrate how the number and stability of bumps depends on n by plotting the 
bump centers on the domain x £ [—it, tt] for various values of n in Fig. 4.1(a). Notice 
that as n is increased, the x = bump reverses its stability at particular values of n. 
This result will be computed in our analysis of linear stability. 

For a more illustrative analysis, we study the case of a Heaviside firing rate 
function (1.5). Under this assumption, we can state the problem of looking for bump 
solutions u(x, t) = U(x) by giving the requirement U(x) > 9 for x G (—a, a) and 
U(x) < 9 otherwise so we can compute the amplitude 

A + = I cos a; + a cos x cos nxdx, (4-5) 

J — a 

and the other amplitude is given 

r 

A- = / cos a; — crcosxcosnxdo:. (4-6) 



27 




Fig. 4.2. Bumps in the ring network (1.1) using heterogeneous synaptic connectivity (1.3) with 
modulation frequency n = 1. (a) Bump half-width a as it depends on amplitude of heterogeneity 
amplitude a. A wide bump centered at x = n (red dashed) separates the wide stable bump at x = 
(blue solid) from itself on the periodic domain. Narrow bump at x = (grey solid) separates stable 
bump from homogeneous off state, (b) Profile of each bump for a = 0.2. Threshold parameter is 
9 = 0.5. Firing rate function is Heaviside (1.5). 



Thus, we can see that switching the sign of a will still yield the same set of bump 
solutions, but they will be centered at different places. 

Now, we compute the bump amplitudes (4.5) and (4.6), which only differ in the 
sign of a. First, we analyze the special case n — 1 , in which A± can be integrated 

A± = 2 sin a ± aa ± — sin(2a) . 

Invoking the threshold condition U (±a) = 0, we can generate and implicit equation 
for the bump half-width a given by 

sin a + sin(3a) 



sin 2a ± a 



(4.7) 



Per our general analysis of the symmetry of bump solutions, we expect there to only 
be one peak location for each sign of a (x = and x = it), since the period of wi in 
this case is 2ir, the length of the domain. However, as in the case of the homogeneous 
weight function, there can be two half-widths a at each location. As we can compute 
using linear stability, a maximum of one bump at each position of these will be linearly 
stable. This is demonstrated in Fig. 4.2. 

In the case that n > 1, we can integrate (4.5) so that we find 



2 sin a ± a 



sin((n — l)a) sin((n+l)a) 



f 



1 



(4.8) 



Upon requiring the threshold crossing conditions U(±a) — A±cosa = 9, we can 
implicitly specify the bump half-width with the equation 

sin((n — 2)a) 2nsin(na) sin((n + 2)a) 



6 = sin(2a) ± 



n-l 



n 2 - 1 



1 



(4.9) 



Since cos a; is a unimodal function, its sole maximum will occur at x = (x = ir/n), 
when Aa- > (A- > 0). Therefore, we do not expect the appearance of multibump 
solutions in this context. We would only expect this if the heterogeneity in (1.3) were 
in the x variable. We now proceed to study the linear stability of the bump solutions 
specified by (4.7) and (4.9). 
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4.2. Stability of bumps. We now study the stability of bumps in the network 
(1.1) with heterogeneous synaptic weights. As we observed in our existence analysis, 
switching the sign of a will lead to the two classes of bumps changing places. There- 
fore, we only study the stability of bumps centered at x — 0, as simply flipping the 
sign of a will provide us with stability of the complementary bump. To analyze the 
stability of the bump, we study the evolution of small, smooth, separable perturba- 
tions to the bump u(x,t) = U(x) + ip(x)e Xt , where \"4>(x)\ <C 1. Plugging into (1.1) 
and truncating to first order 



(A + l)ip(x) = [ w(x,y)f'(U(y))iP(y)dy. (4.10) 

J —71 

The essential spectrum A = — 1 docs not contribute to any instabilities here. There- 
fore, any instabilities can be identified by studying the point spectrum. We can 
identify these by appropriately manipulating the integral term in (4.10). In the case 
of the particular weight function (1.3), we can apply the identity (2.2) to write the 
equation (4.10) as a 2 x 2 linear spectral problem 



(A + l) 

where 



A \ _ / J n (cos 2 x) J„(cosxsinx) \ / A 
B ) I J~ n (cosa:smx) J~„(sin 2 x) J \ B 



Jn(r(x))= f r(x){l+acos{nx))f'(U(x))dx. 

J —J! 

It is clear that, since U(x) is even, J n (cos x since) = 0. Therefore, the two eigenvalues 
describing the linear stability of the bump (4.1) will be one associated with odd 
perturbations 

A = -1 + J~„(sin 2 x) 

and one associated with even perturbations 

A e = -1 + J„(cos 2 x). 

In the case of a Heaviside firing rate function (1.5), we can use (2.22) to calculate 
the integral terms 

, N 2 sin 2 a + 2a cos(na) sin 2 a 

J„(sm 2 x) = ^ — 

A+ sin a 

„ , 9 , 2 cos 2 a + 2cr cos(na) cos 2 a 

J n COS 2 X = r-^-^ . 

A + sin a 

To study the effect of heterogeneities on the eigenvalues, we start with the special 
case n — 1. Here, the eigenvalue associated with odd perturbations is given 

2 sin a + 2a sin a cos a aa „ , , _ , 

X o = -l + 7T-- S : = -7T-- r^<°; 4 - n 

2 sin a + aa + 2a sm a cos a 2 sin a + aa + a sin 2a 



since a > 0. Thus, we can be certain that the bump is linearly stable to shift pertur- 
bations when n = 1 and a > 0. In a complementary way, bumps in the network where 
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Fig. 4.3. Eigenvalue X a associated with odd perturbations of the bump centered at x = given by 
(4-1) with amplitude A+ specified (4-5). (a) Eigenvalue A_ as a function of heterogeneity amplitude 
a becomes negative, indicating linear stability, when n = 1 and n = 2 but become positive, indicating 
linear instability, when n = 4. (b) Eigenvalue \ as a function of synaptic modulation frequency n 
as determined by the formulae (4-11) for n = 1 and (4-13) for n > 1. Heterogeneity amplitude is 
fixed a = 0.2. Threshold parameter 8 = 0.5. 



a < will be linearly unstable to shift perturbations when n = 1. Even perturbations 
have associated eigenvalue 



A e = — 1 + cot a 



> 2 sin a + 2a sin a cos a 



2 sin a + aa + 2a sin a cos a 

For n > 1, the eigenvalue associated with odd perturbations will be 

1 + a cos na , J _ „. 

A = -l + — F — — — -, (4.12) 

1 + [n cot a sm(na) — cos(na)J 



an[n sin a cos(rta) — cos a sin(na)] 
(n 2 — 1) sin a + a[n cos a sin(na) — sin a cos(na)] 



(4.13) 



which will, in general, not be zero. We plot the eigenvalue A Q as a function of a and 
of n in Fig. 4.3. As we have mentioned, the eigenvalue A oscillates as a function of n 
so that the bump at x = reverses its stability. The eigenvalue associated with even 
perturbations of the bump will be given by 

9 1 + a cos na 

A e = — 1 + cot a- 



1 + [n cot a sin(net) — cos(na)] 

In general, its sign will not change for small amplitudes a. 

4.3. Pinned bumps for low frequency modulation. To analyze the effect 
that noise has upon bump solutions, we will begin by making a small noise assump- 
tion and performing an asymptotic expansion, as we did for the homogeneous net- 
work. Due to spatial heterogeneities, noise causes the center of the bump to move as 
a mean-reverting stochastic process, rather than a purely diffusive process. Synaptic 
heterogeneities, however subtle, can trap neural activity in basins of attraction whose 
widths are defined by the period of the heterogeneity (1.3). On exponentially long 
timescales we would expect the bump to escape from these potential wells. However, 
even for weak heterogeneities, escape rates are low enough such that the movement 
of the bump away from its initial condition can occur more slowly than in the homo- 
geneous case. 
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Our analysis here is mainly concerned with the effect periodic heterogeneities 
have upon the diffusion of bumps. Thus, we merely consider the case of additive noise 
(g(U) — 1), so the noise will have zero mean. Though the case of multiplicative noise 
could be analyzed, that of additive noise makes for more transparent results. We 
assume the additive noise in (1.6) generates two phenomena that occur on distinct 
timescales. The center of the bump will fluctuate about its original position on long 
time scale according to the stochastic variable A(t). On short timescales, the profile of 
the bump U will fluctuate according to the series of higher order corrections e 1 / 2 ^' + 
£$i + e 3//2 $ 2 + • • • • Thus, we plug the expansion (3.18) into (1.6) and study the 
hierarchy of equations generated by expanding in powers of e 1 / 2 . To leading order, 
we find the deterministic equation (4.1) for the mean bump profile U(x). To next 
order, we find that A(t) = 0{e 1/2 ) and 

d$(x, t) = £${x, t)dt + e- 1/2 U , (x)dA(t) + dW(x, t) + e- 1/2 B(x)A(t)dt (4.14) 

where C is the non-self-adjoint linear operator (3.20), and 

B(x)=anf w[(ny)w(x-y)f{U(y))dy. (4.15) 
J —n 

The last term on the right hand side of (4.14) is generated by integrating the hetero- 
geneous contribution from the weight function (1.3) by parts and truncating with a 
linearization. Notice that since B(x) scales with n, this approximation will only be 
valid for small enough n values. Thus, we only consider the effect of low modulation 
frequencies n in this subsection. Now, we can ensure that a bounded solution to (4.14) 
exists by requiring the inhomogeneous part is orthogonal to the nullspace ip(x) of the 
adjoint operator C* defined by (3.21). Upon taking the L 2 inner product of both sides 
of (4.14) with ip(x), we have the solvability condition 

f (p(x) \u'(x)dA(t) + B(x)A(t)dt + e 1/2 dW(x, i)l da; = 0. (4.16) 

J — 7T 

We can then rearrange the stochastic differential equation (4.16) to find that A(f) 
satisfies the Ornstein-Uhlenbeck process 

dA(i) + KA(t)dt = dW(t), (4.17) 

where 

K - r_ w <p(xw'(x)dx (4J8) 

and W(t) is the white noise process defined by (3.25) having diffusion coefficient D(e) 
(2.39) with g{U) = 1, as in the case of the input-driven network (3.1). Assuming 
we start the bump upon a stable attractor, as defined by our existence and stability 
calculations of (1.1) with synaptic weight (1.3), we can calculate the mean and variance 
of the Ornstein-Uhlenbeck process (4.17) using standard techniques [31] to find 

{Ait)) = 0, (A(t) 2 ) - (A(t)) 2 = [1 - e- 2Kt ] . (4.19) 

This provides us with a different result than the translationally symmetric system 
with w(x,y) — w(x — y) where the bump freely diffuses. However, it is related to 
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Fig. 4.4. Pinning of bumps in the network (1.6) with synaptic weight (1.3) for low freguency 
n synaptic heterogeneity, (a) Numerical simulation of (1.6) using synaptic weight (1.3) for n = 2, 
o~ = 0.1, and e = 0.01 shows bump remains pinned to the stable attractor at x = 0. (b) Variance of 
the bump's position plotted against time computed numerically (red dashed) across 1000 realizations 
saturates after a moderate amount of time when n = 2, as predicted by the Ornstein- Uhlenbeck 
approximation (4-17) (blue solid), (c) Numerical simulation for n = 3, a = 0.1, and s = 0.01 shows 
bump remains pinned to the stable location at x = 0. (d) Variance of the bump's position plotted 
against time computed numerically (red dashed) does not match the prediction of the Ornstein- 
Uhlenbeck approximation (blue solid) guite as well for long times. Threshold parameter 8 = 0.5 



the result we found in our input driven system (3.1) where an external input (3.2) of 
frequency n locks the bump to the vicinity of a n discrete attractors. Here the bump 
is pinned by internal bias generated by the heterogeneous contribution of the weight 
kernel w\{ny), so that the variance saturates at D(e)/ n for large values oft, according 
to the approximation (4.19). 

In order to study our asymptotic analysis, we consider the case of a Heaviside firing 
rate function (1.5), cosine (1.2) for w and w\, and cosine spatial noise correlations 
(2.47). In this case, the diffusion coefficient D(e) is given by the formula (2.49). In 
addition, we restrict our modulation frequency to be greater than unity, n > 1. Then 
the function B{x), which leads to pinning, can be computed, using the formula (2.45) 
for f(x) so 

f a . cosasin(na) — nsinacos(na) . 
B(x) = —cm / sm(ny) cos(x — y)ay = —2an 5 smi. 

J -a n - 1 

Plugging this into our formula for the mean reversion rate (4.18), where we use our 
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formula for the amplitude of the bump (4.8), we have 

an [cos a sin(na) — n sin a cos(na)] 



(4.20) 



(n 2 — 1) sin a ± cr[n cos a sin(na) — sinacos(na)] 



which, not surprising, is simply the eigenvalue A_ associated with odd perturbations 
(4.13), up to a sign switch. The sign of the a portion of the denominator is am- 
biguous because we must select the stable bump, which could have either A + or 
A_ as its amplitude. Using these specific formulae, we can compute the variance of 
the Ornstein-Uhlenbeck process (4.17) with the formula (4.19). We show example of 
this in Fig. 4.4 for n — 2 and n — 3. In particular, we observe that the variance 
of the bump, computed by averaging across many realizations of (1.6) saturates af- 
ter a substantial amount of time. However, as the number of attractors is increase, 
the Ornstein-Uhlenbeck approximation (4.17) does not do as well approximating the 
variance, since the bump can begin to escape from the starting pinned location to a 
neighboring one. 

4.4. Reduced diffusion of bumps for high frequency modulation. As op- 
posed to the expansion we performed in the previous subsection, we could consider a 
perturbative approximation that takes into account the nonlincarity of the synaptic 
heterogeneity (1.3), rather than linearizing it to yield the Ornstein-Uhlenbeck approx- 
imation (4.17). To do so, we can take note of the fact that, as n becomes large, the 
contribution made by the heterogeneous part of (1.3) becomes small. Thus, it is not 
necessary to perform an expansion of this portion in A(t) in order to truncate the 
integral term in (1.6). In fact, doing so would cause ever worse approximation, due 
to the slope of the linearization (4.15) becoming steeper and steeper, as it scales with 
n. This is related to the fact that as n increases, the bump begins to escape from the 
vicinity of individual discrete attractors more often (see Fig. 4.5). As before, we can 
perform the expansion (3.18) where U is a stationary bump solution (4.1), A(i) tracks 
the wandering of the bump, and $ tracks fast fluctuations in the profile of the bump. 
Plugging this into (1.6), we perform a similar averaging and hierarchy expansion to 
before. The 0(e 1 ^ 2 ) equation is then given 



d*(a;, t) = C$(x, t)dt + e- 1/2 U'{x)dA{t) + dW(x, t) + e- 1/2 B{x, A(i))dt, (4.21) 



which we will show to be small below. To derive the function B(x, A), we have 
performed the change of variables 



where C is the non-self-adjoint linear operator (3.20), and 




(4.22) 





— 7T 



(1 + aw 1 (ny))w{x - A - y)f(U(y))dy 




{w 1 (n(y + A)) - wx(ny))w(x- A - y)f{U{y))dy 1 



in order to make the cancellation 



U(x - A) = 




— 7T 



(1 + Gw 1 {ny))w{x - A - y)f(U{y))dy. 
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Since W\{ny) is a 27r/n-periodic function, we can also assume that B(x,A) will be 
27r/n-periodic in A. To justify the retention of the term B(x, A) in the 0(e 1 ^ 2 ), we 
note that upon integrating (4.22) by parts, we have 

B(x, A) = - ^ W d {ny)^- [w(x -A- y)f(U(y))} dy = 0(l/n), 

which will be small for large n, and we have defined 

W d (x) = I [wi(y + A) - wi(y)] dy. 

J —tt 

Note also that since we require A(t) = 0(e 1 ^ 2 ) for our approximation, we can truncate 
a Taylor expansion of w(x — y — A) so that we define 

[ Wl (n(y + A)) - Wl (ny)} w{x - y)f(U(y))dy. (4.23) 

Now, we can ensure that a bounded solution to (4.21) exists by requiring the inhomo- 
geneous part is orthogonal to the nullspace (p(x) of the adjoint operator C* defined 
by (3.21). Upon taking the L 2 inner product of both sides of (4.21) with (p(x), we 
have the sufficient solvability condition 



/: 



<p(x) U'(x)dA(t) +B(x,A)dt + e l/2 dW{x 1 t) dx = 0. (4.24) 



We can then rearrange the solvability condition (4.24) to find that A{t) satisfies the 
nonlinear stochastic differential equation 

dA(i) + K{nA)dt = dW(t), (4.25) 

where 

r v{x)B(x, A)dx 

is a 27r/n-periodic function since B(x, A) is 27r/n-periodic in A, and the white noise 
process W(t) still defined by (3.25) having diffusion coefficient D(e) given by (2.39) 
with g{U) = 1, as before. Therefore, we have reduced the problem of a bump wan- 
dering in a neural field with periodic synaptic microstructure to that of a particle 
diffusing in a periodic potential. This is a well studied problem for which it is pos- 
sible to derive an effective diffusion coefficient [61]. To do so, we must derive the 
profile of the periodic potential well governing the dynamics. To find this, we simply 
integrate the nonlinear function (4.26), which yields 

f A r ip(x) f A B(x,-n)dndx 

"<*>-/_ *("»»>- J -"^;j ( ;v- M l • ( " 7) 

With the 27r/n-periodic potential well (4.27) in hand, we can derive the effective 
diffusion coefficient 

Ptf/= ,- <*">■>; (*('»' , 4 ,8, 

t— >oc t 



34 



of the stochastic process defined by (4.25). As the definition of D e fj (4.28) suggests, 
the approximation is valid in the limit of large time. However, we do find that it works 
quite well for reasonably short times too. This is contingent upon the modulation fre- 
quency n being substantially large. As many authors have found, this approximation 
arises from the fact that the density of trajectories tends asymptotically to [50, 51] 



P as (A,t) =P (A) 



exp[-A 2 /4£> e// t] 
y/4nD eff t 



(4.29) 



where Pq refers to the stationary (27r/n-periodic) solution of (4.25). This function 
is responsible for microstructure of the density whereas the Gaussian is responsible 
for it macrostructure. Usually, this structure is numerically extracted by evolving the 
Fokker-Planck formalism of the Langevin equation (4.25), so the approximation (4.29) 
can be made as an ansatz. In this case, we can approximate using the Lifson- Jackson 
formula [50, 26, 61] 



D, 



eff 



D(e)(2n/n) 



/o^iT^xp 



2(V(x)-V{y)) 
D(e) 



dydx 



(4.30) 



where we have used the diffusion coefficient D(e) of the white noise source and the 
27r/n-periodicity of the potential well (4.27). As we will show, the heterogeneity 
introduced in the synaptic weight (1.3) tends to decrease the effective diffusion co- 
efficient. In other words, we usually find that D e ff < D(e). Thus in some sense, 
having a chain of discrete attractors appears to provide better memory of the initial 
condition than a line attractor. Of course the trade off is that only a finite number 
of initial conditions, specifically n, can be represented in our network (1.6) with the 
weight (1.3) with modulation frequency n. We will explore this issue further in future 
studies. 

For now, we compare the asymptotic approximation of D e f f to numerical simula- 
tions. We thus consider the case of a Heaviside firing rate function (1.5), cosine (1.2) 
for w and W\, and cosine spatial correlations (2.47). In this case, the diffusion coef- 
ficient D(e) is given by the formula (2.49). First, we compute the function B(x, A) 
(4.23), which is 27r/n-periodic in the A argument 



B(x, A) = a 



2(cos(nA) — l)(ncosasin(na) — sinacos(no)) 



n 2 - 1 

2 sin(nA) (n sin a cos(na) — cos a sin(na) ) 



cosx 



n 2 -l 



sin a;. 



(4.31) 



Now, with the formula (4.31) in hand, as well as (2.45) for ip(x) and the equation for 
the amplitude A± (4.8), we can compute the nonlinear function K(nA) using (4.26). 
Note that the cosine portion of (4.31) vanishes upon integration to yield 



K(nA) = 



2a(nsmacos(na) — cos a sin(na)) 
(n 2 — 1) sin a ± a[n cos a sin(na) — sin a cos(na)] 



sin(nA), (4.32) 



where we select the + or — in the denominator of (4.32), depending on whether 
the bump centered at x = is stable or not. Now, in order to compute our effective 
diffusion coefficient D e ff, we must integrate the function K(nA) to yield the potential 
function governing the dynamics of (4.25). This gives us the potential function 



V(A) = -V(n)cos(nA), 



(4.33) 
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Fig. 4.5. Reduced effective diffusion in a network with high frequency modulation in synaptic 
weights, (a) Numerical simulation of (1.6) with synaptic weight (1.3) in the case n = 8 where 
the bump makes frequent jumps between locations of stable attractors (cyan) of the deterministic 
system, (b) Variance in the bumps position scales linearly with time, rather than saturating as in 
the case of lower frequency modulation of synaptic weights, (c) Probability density P(A,t) of bump 
position computed across 5000 realizations evaluated at time t = 400 reveals microperiodic structure 
of diffusion suggested by (4-29). Vertical lines (cyan) indicate the location of the n = 8 attractors. 
(d) Asymptotic approximation of effective diffusion D e ff (blue line) computed using theory (4-35) 
as compared with that computed using numerical simulations (red dashed dot). For small values 
of n, effective diffusion is considerably reduced as compared to diffusion (2.49) in the homogeneous 
system (black line). Other parameters are 8 = 0.5, a = 0.1, and e = 0.01. 



where the amplitude (or half- height) of each well is 

2a(n sin a cos(na) — cos a sin(na)) 



V(n) 



n(n 2 — 1) sin a ± an[n cos a sin(na) — sinacos(na)] 



(4.34) 



Now, finally, we use the standard formula for the effective diffusion coefficient of a 
particle in a periodic potential well (4.30). With our particular cosine potential well 
(4.33), we find that each integral can be computed and are equal 



2-k/t, 



exp 



2V(x) 
~D(e) 



Ax = 



2-n/r. 



exp 



V(x) 



exp 



D(s) 

2V(n) 
D{e) 



d.T 

cos(na:) 



A 2lT T 

ax = — io 

n 



2V(n) 
D(s) 



where Iq(x) is the modified Bessel function of the zeroth kind. Therefore, the formula 
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(4.30) for the effective diffusion coefficient yields 

Deff = [I (2V(n) £ /D(eW (435) 

Using this formula along with the definition (4.34), we approximate the diffusion of a 
bump in a network with synaptic modulation frequency n = 8 in Fig. 4.5(b). Notice, 
the linear approximation of the variance's scaling with time matches averages over 
realizations fairly well. Thus, the variance no longer saturates in time, as in the 
case of low frequency modulation n. As evidenced by our plots of the probability 
density P(A,t), in Fig. 4.5(c), the stochastic process A(t) behaves diffusively with 
microperiodic modulation, as suggested by the asymptotic formula (4.29). Now, we 
can note that in the limit of high amplitude modulations (n — > oo) the formula (4.35) 
tends to the diffusion coefficient of the homogeneous network since 

, , , 2(7(nsinacos(na) — cosasin(na)) 

hm V (n) = lim 1 V > \" — - = 

— n(n z — 1 ) sin a ± an[n cos a sm(na) — sin a cos(na)\ 



lim M^rr) =M0) = i, 



so that 

lim Tn I 

D(e 

and thus we find the limit of (4.35) to be 

lim D eff = D(e). 

n— >oo 

Since Iq{x) has a global minimum at x = 0, it is clear that -D e // < D(e) for all n. 
However, in numerical simulations, we would also presume the effects of pinning, as 
described by (4.17) would also be present. Nonetheless, we compare our theoretical 
effective diffusion (4.35) across a span of modulation frequencies n to that approx- 
imated using numerical simulations in Fig. 4.5 (d). We find reasonable agreement. 
In particular, we see the result that synaptic heterogeneity substantially reduces the 
effective diffusion of the bump for lower values of n. We plan to pursue this result 
much more deeply in future studies. 

5. Discussion. We have analyzed the effects of external noise on stationary 
bumps in spatially extended neural field equations. In a network with spatially ho- 
mogeneous synaptic weights, we found that noise causes bumps to wander about 
the spatial domain according to a purely diffusive process. We can asymptotically ap- 
proximate the diffusion coefficient of this process using a small-noise expansion, which 
assumes the profile of the activity variable is still a bump to first order. Comparing 
the effects of purely additive and multiplicative noise, we find that multiplicative noise 
alters the mean amplitude of the bump profile while additive does not. Following this 
analysis, we study the effects of breaking the translation symmetry of the spatially 
homogeneous network in two ways, using external inputs and using spatially heteroge- 
neous synaptic weights. Effectively, this alters the dynamic landscape of the network 
from a line attractor to a chain of discrete attractors. External inputs with multiple 
peaks serve to pin the bump to one of multiple discrete attractors of the network, 
so that the bump's position evolves as a mean-reverting process. Periodic synaptic 
heterogeneity also leads to pinning at low modulation frequencies. At high modula- 
tion frequencies, the bump can escape from being pinned to a single location in the 
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network, leading to effective diffusion in the limit of long time. We can approximate 
this effective diffusion using methods for studying a particle diffusing in a periodic 
potential. 

We see the main contribution of this work as introducing the notion of reliability, 
in the presence of noise, to stationary bumps in neural fields. The specific location of 
a bump in a neural field carries important information about the stimulus that formed 
it [1, 12, 48]. Noise can degrade this memory, so it is very useful to understand how the 
architecture and parameters of a neural field model affect how easily this deterioration 
takes place. This has specific applications in the realm of oculomotor delayed-response 
tasks in prefrontal cortex, where it is clear there are networks of neurons that can 
encode visuospatial location during the retention period of such tasks [29, 32, 11]. 
Since our work shows that breaking the translation symmetry of neural fields can 
serve to decrease noise-induced diffusion of bumps, it is worth pursuing how well 
this improves the overall memory process. The advantage of a network that is a 
line attractor is that, in the absence of noise, it can represent a continuum of initial 
conditions. Since all of these representations are marginally stable, memory is easily 
degraded when in line attractors when noise is introduced. On the other hand, when 
symmetry is broken so a network behaves as a chain of discrete attractors, there is a 
trade-off between initial representation errors versus long term robustness to noise. 

Neural fields are known to generate a variety of spatially structured solutions 
other than bumps, such as traveling waves [71, 1, 5, 7, 21], stationary periodic patterns 
[36, 22, 62], and spiral waves [35, 8]. It would be interesting to study more about how 
these structures are affected by external noise. It seems that the form of the spatially 
structured solution markedly contributes to the the way in which noise affects its form 
and position. Neural fields that support spatially periodic patterns can have the onset 
of the associated Turing instability shifted by the inclusion of spatially structured noise 
[37]. In recent work on traveling fronts in stochastic neural fields, it was found that 
the bifurcation structure of the neural field determined the characteristic scaling of 
front location variance with time [10]. In particular, pulled fronts have subdiffusive 
variance scaling, as opposed to diffusive variance scaling of a front in a bistable system. 
We plan to study the effects of noise on bumps in planar neural fields. In this case, 
the spatial correlations of the noise will be in two dimensions. Therefore, dimensional 
bias in the synaptic weight or noise correlations could lead to asymmetric diffusion of 
the bump in the plane. In addition, it is possible this analysis could be extended to 
two component system, such as a model with local adaptation that generates traveling 
pulses [57]. If there is a separation of timescales between the activity and adaptation 
variable, fast-slow analysis might be paired with the small-noise expansion (2.28) to 
derive the effective variance in position of the traveling pulse. Finally, it would be 
quite interesting to study the effects of noise on spiral waves in neural fields [35]. 
Doing so may provide us with some experimentally verifiable measure of whether 
long-time deviations of the spiral center arise from deterministic meandering or noise. 
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